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EXECUTIVE SUMMARY 



Study Overview 

This study identifies community college management practices that promote student success. It 
was conducted by the Community College Resource Center (CCRC) through a partnership with 
the Florida Department of Education’s Division of Community Colleges and Workforce 
Education, and funded by the Lumina Foundation for Education as part of the Achieving the 
Dream: Community College Count initiative. Achieving the Dream is a national effort to 
increase the success of community college students, particularly those from groups that have 
been underserved in higher education. The initiative works on multiple fronts - including 
technical assistance to individual community colleges, research, public engagement, and public 
policy - and emphasizes the use of data to drive change. 

This study builds on earlier CCRC research using national survey data. We used transcript-level 
data on 150,000 students in three cohorts of first- time Florida community college students and a 
regression methodology to estimate the effect that each of Florida’s 28 community colleges had 
on the probability that its students would achieve a successful outcome, after controlling for 
characteristics of the individual students. This effect can be seen as a measure of value added - 
the impact that a college has on its students’ educational success independent of the 
characteristics of individual students. We then ranked the colleges according to their estimated 
effects on student success. 

Given the interest of Achieving the Dream in underserved students, we selected colleges for field 
research using rankings of the magnitude of the effect of each institution on the probability that 
its African American and Latino students would attain successful outcomes. In Florida, as in 
other states, African American and Latino community college students are less likely than other 
students to complete a degree or to transfer to a baccalaureate program. At the same time, 
because we are interested in what colleges are doing to retain students generally, we also 
examined each institution’s impact on outcomes for all first-time students. 

We used these rankings along with an analysis of descriptive statistics on each institution to 
select six colleges for field research: three with higher impacts on the chance that their minority 
students would succeed and three with lower impacts. The purpose of the fieldwork was to 
compare the institutional policies, practices, and cultural characteristics of the high- and low- 
impact colleges during the period in which the student cohorts were tracked (from academic year 
1998-1999 through 2002-2003) to determine why some colleges had a greater net effect on their 
minority students’ educational success than did others. 



Research to Date 

There has been surprisingly little rigorous research on institutional effectiveness in community 
colleges. Even the much larger body of research on institutional effectiveness among four-year 
institutions tells us more about the student characteristics and general institutional features (e.g., 
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selectivity, size, resources) associated with positive student outcomes than about the policies and 
practices affecting student success that are under a college’s control. A key problem in this 
research is the difficulty in comparing the performance of different institutions serving different 
mixes of students. 

Recent studies have sought to examine the policies and practices of undergraduate institutions 
that perform better than would be expected given their students’ characteristics. While these 
studies offer important insights into the elements of institutional effectiveness, they focus on 
four-year institutions, so the applicability of their findings to community colleges is 
questionable. They also suffer from a number of data and methodological limitations. 

The study of institutional effectiveness in community colleges summarized here addresses the 
limitations of previous research on the effectiveness of undergraduate institutions in several 
ways. It takes advantage of a rich set of longitudinal student unit record data to control for the 
individual characteristics of the students that the colleges serve. Because our study is based on 
the outcomes of both full-time and part-time students, its measure of institutional effectiveness is 
better suited to community colleges and their students than is the National Center for Education 
Statistics’ (NCES) “student-right-to-know” measure commonly used by other studies. We also 
measured student persistence in addition to completion and transfer, which is appropriate given 
that community college students often take a long time to complete their programs or to transfer. 
Our sample is confined to all community colleges in a single state, thus eliminating the effects on 
institutional performance of variations in public policy and institutional mission, practice, and 
resources across states. 

While some previous studies examined only institutions considered to be high performers, we 
directly compared colleges found to have a relatively high impact on the educational success of 
their students with colleges that have a low impact. In studying the policies and practices of the 
high- and low-impact colleges, we focused on their organization, activities, and environments 
during the period covering our data (1998-2003). Other studies have not addressed the fact that a 
college’s policies and practices can change over time. 

As noted, most of the earlier research on the effectiveness of undergraduate institutions relied on 
conceptual frameworks drawn from the model and experience of four-year colleges, particularly 
residential colleges. For this study, we developed a conceptual framework that reflects the 
distinctive challenges and characteristics of community colleges. 



Research Hypotheses 

To frame this study of community college effectiveness, we drew on previous research to 
develop a set of seven hypotheses about the ways that we expected that those community 
colleges more effective in promoting their students’ education success would differ from those 
with a less positive impact on student success. Specifically, we hypothesized that community 
colleges would be more effective if they do the following: 
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Have an institutional focus on student retention and outcomes, not just on enrollment. We 
expected that effective community colleges would be actively focused on student retention and 
outcomes, while less effective colleges would be primarily concerned about enrollment. Funding 
for community colleges (and for public higher education institutions generally) is largely based 
on enrollments rather than on completions or other outcomes, so it is not surprising that 
community colleges often pay more attention to the former than the latter. 

Offer targeted support for underperforming students. Where there are systemic gaps between 
the outcomes of different student groups, as is typically the case between minority and White 
students, we expected that effective colleges would undertake targeted efforts to address these 
gaps. 

Have well-designed, well-aligned, and proactive student support services. We hypothesized that 
effective community colleges would offer student services that are well designed and aligned to 
guide and support students from the time they enter the college until they leave. Effective 
colleges use technology and other means proactively to identify and reach out to students who 
are struggling or are at risk of dropping out to ensure that they stay on track. 

Provide support for faculty development focused on improving teaching. We expected that 
effective colleges would devote more attention and resources to helping faculty become better 
teachers, with a particular focus on instructors of students who are academically unprepared or 
from minority populations. Given that the majority of faculty in most community colleges are 
part time, we hypothesized that effective colleges would take steps to orient and prepare adjunct 
instructors and monitor the quality of their teaching. 

Experiment with ways to improve the effectiveness of instruction and support services. We 
expected that effective community colleges would be more likely than less effective colleges to 
experiment with better ways to teach and support students, particularly those who have academic 
deficiencies. Because so many community college students are required to take remedial or 
developmental courses, and many students who take such courses fail to advance to and succeed 
in degree credit coursework, we expected that effective colleges would to try to find ways to 
improve the outcomes of developmental programs. We also expected that effective colleges 
would be more likely to evaluate the impact of efforts to improve student learning and success 
and to use the findings to inform further improvements in practice. 

Use institutional research to track student outcomes and improve program impact. We 
hypothesized that effective community colleges would collect data on student outcomes and use 
this information to evaluate and manage programs and services in ways that improve student 
success. This hypothesis is based on the premise of organizational management that “you 
measure what you value,” and its corollary, “you don’t value what you don’t measure.” A recent 
survey by the Community College Research Center on community college institutional research 
found that relatively few colleges collect data on student outcomes, other than what is required 
for compliance and accountability. Even fewer colleges use data on student outcomes to inform 
decisions about how to organize and deliver programs and services. 
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Manage the institution in ways that promote systemic improvement in student success. Our 
final hypothesis was that the more effective community colleges would approach institutional 
management with a strategic focus on improving student outcomes. Specifically, we expected 
effective colleges to have in place systems, policies, and procedures for program review, 
strategic planning, and budgeting that are guided by evidence of what works to promote student 
success and that are designed to foster systemic improvements in the impact on students of the 
college’s academic programs and student services. 

Taken together, the elements of institutional policy, practice, and culture defined by these 
hypotheses form a model of community college institutional effectiveness that we tested through 
this study. What connects our hypotheses is the idea that effective community colleges 
deliberately and systematically manage programs and support services in ways that optimize the 
impact of the college’s limited resources on student success. Making effective use of resources is 
especially important to community colleges because a high proportion of community college 
students are poorly prepared for college and therefore more costly to serve than well-prepared 
students, and yet community colleges have relatively few resources to serve their students 
(compared with four-year institutions). From this perspective, a college’s effectiveness in serving 
students results less from whether it adopts particular policies or practices than from how well it 
aligns and manages all its programs and services to support student success. 



Research Design 

We used longitudinal, transcript-level data on 150,000 students in three cohorts of Florida 
community college students to estimate the effect that each of the 28 community colleges in the 
state has on the probability of its students’ completing a certificate or degree, transferring to one 
of the state universities, or persisting at the college. The cohorts consisted of first-time college 
students who enrolled in a degree program at a Florida community college in the fall of 1998, 
1999, or 2000. 

We ranked all 28 Florida community colleges according to the size of their respective estimated 
effect on the probability that entering minority students would complete, transfer, or persist 
within three years. The college with the highest average effect was ranked first; the one with the 
lowest effect was ranked last. For the field research, we selected three “high-impact” colleges 
from the top-ranked institutions and three “low-impact” colleges from the low-ranking 
institutions. 

Table 1 compares the three high-impact colleges selected with the three low-impact colleges 
according to their quartile ranking among all 28 Florida community colleges on selected 
institutional characteristics. The high-impact and low-impact colleges are roughly comparable on 
characteristics that other studies have found to affect student outcomes and institutional 
performance and for which we do not control in our selection methodology. They include such 
factors as instructional expenditures, proportion of faculty who are part time, and the ratio of 
certificates to associate degrees. In fact, the low-impact colleges rank higher in instructional 
expenditures and are somewhat more oriented toward awarding occupational certificates over 
degrees, which might give them an advantage over the high-impact colleges in student 
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completion rates. Two of the high-impact colleges have higher proportions of part-time students, 
which might put them at a disadvantage. The low-impact colleges include no urban institutions 
or any very large colleges, although the high-impact colleges include a small institution. 
Therefore, even though we were unable to control statistically for the effects on performance of 
institutional characteristics, there do not seem to be substantial differences between the sets of 
high- and low-impact colleges that might explain their relative performances. 

We conducted interviews and reviewed documents at the six sample colleges. The goal of this 
field research was to identify differences in the policies and practices of the high-impact and 
low-impact colleges that would explain why, controlling for individual student characteristics, 
minority students at the former were more likely to graduate, transfer, or persist than were 
minority students at the latter. Since the data used to select the two sets of institutions covered 
the period 1998 to 2003, our field research focused on the colleges’ policies and practices during 
that period. 



Findings and Conclusions 

Table 2 summarizes the state of development of the seven elements of the hypothesized model of 
institutional effectiveness at the six colleges we visited, indicating for each college whether the 
given element was well developed, developing, or weak or non-existent during the study period. 

Our findings indicate that the dimensions of our model of community college effectiveness 
where there is the clearest difference between the high- and low-impact colleges fit into the 
category, “Targeted support for minority students,” and specifically into “Minority-inclusive 
campus environment” and “Specialized retention services for minority students.” Thus, minority 
community college students are more likely to succeed at colleges where they are made to feel 
welcome and where there are support services and programs specifically designed for them. 

At the three low-impact colleges (identified as L-I A, L-I B, and L-I C), and at one of the high- 
impact colleges (H-I C), some respondents argued that community colleges should not give 
preferential treatment to any one group because many if not most community college students 
face barriers to success in college. Others at these four colleges, and particularly some (but not 
all) of the minority staff and faculty we interviewed, maintained, conversely, that special efforts 
are needed precisely because there are persistent gaps in achievement between minority and 
White students. The findings from this study support this latter position. 

Most of the other dimensions of the model beyond those related to a focus on minority student 
achievement were better developed at two of the high-impact colleges (H-I A and H-I B) than at 
the three low-impact colleges. This finding supports our hypothesis that policies and practices 
that affect students generally can benefit minority students as well as others. These practices 
include a focus on student retention and graduation, rather than just on enrollment; well-aligned 
and proactive student support services; experimentation with ways to improve student success; 
and use of data on students to improve programs and services. 
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H-I C seems to be a conflicting case. However, further analysis shows that, while H-I C ranks 
high among Florida community colleges in its impact on minority student success (adjusted for 
student characteristics), it ranks low in its effect on success of students generally. The other two 
high-impact colleges (H-I A and H-I B) rank high in their impact on the adjusted success rates of 
all students, while the three low-impact colleges have relatively low impacts on overall student 
success. 

The three small colleges in the our sample (H-I C, L-I B, and L-I C) generally lacked formal 
student support services. They relied instead on the commitment of faculty and staff, and the 
personal attention made possible by their smaller size, to create a supportive environment for 
students. Respondents at each of these colleges argued that these college qualities were more 
important to student success than formal systems, procedures, and programs. The findings here 
do not support this contention, however. 

Because it was categorized as a high-impact institution, but lacked formal systems, procedures, 
and programs for student support, H-I C once again seems to present a conflicting case. Yet, 
further investigation shows that most of the minority students at the college during the study 
period were likely to have benefited from the more organized and intensive support services 
provided through a federal Student Support Services grant for disadvantaged students. 

Our findings suggest that, to promote student success, not only do particular student support 
services - such as in-depth orientations, proactive advising, early warning systems, and well- 
organized academic support services - need to be in place, but they must be well aligned and 
coordinated across the campus. While administrators may see different functional areas of the 
college as providing discrete services, students do not see, nor should they experience, such 
divisions. Seamless integration of services from the student’s perspective and collaboration 
among faculty, staff, and administration in providing these services are the college characteristics 
that seem to contribute most to student success. 

The findings also support our overarching hypothesis that the key to a college’s effectiveness is 
not whether it adopts particular policies or practices, but how well it aligns and manages all of its 
programs and services to support student success. Small-scale “boutique” programs or pilots may 
represent important sources of innovation for a college in the long term, but they are unlikely by 
themselves to have much of a direct impact on overall institutional effectiveness. 

Of the six colleges, only H-I A had well developed systems and procedures for managing 
college-wide improvements in practice based on research and data on student achievement. They 
included a standing committee responsible for monitoring and evaluating the college’s efforts to 
promote student success, a process for regularly and systematically reviewing the performance of 
all programs down to the course level, and a strategic online planning tool that tracks budget 
requests from each unit to ensure that they are connected with the unit objectives. Each unit’s 
objectives are tied to the college’s goals, which in turn are aligned with state accountability goals 
and measures. Arguably, such an approach is needed to bring about improvements in practice 
over time, although it may not be necessary to create the conditions for student success. 
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In the three years since the end of the study period, all of the six colleges we visited have 
adopted a fuller set of the elements of our model of institutional effectiveness. The shaded cells 
in Table 3 show particular features of the model that are now more strongly developed at each 
college than at the end of the study period. All have sought to strengthen and better align student 
services. As part of these efforts, the colleges have put more information for students on the 
Internet, allowing students to seek answers to their routine questions and freeing up staff to help 
students with more complex issues. All but one of the colleges have strengthened their systems 
for evaluating and improving practices based on student performance data. With one exception, 
all have implemented or are in the process of putting in place management systems to support 
ongoing improvements in programs and services. With these changes since the study period, we 
would expect to see improvements in student success at all six colleges, although data are not yet 
available to allow us to test this prediction. 

In two cases, the impetus for these changes came from new leadership that brought a greater 
focus on improving student outcomes and putting in place systems, policies, and procedures to 
accomplish this goal. In at least two others, it was a response to accreditation reviews indicating 
the need for a more systematic and systemic approach to ensuring institutional effectiveness. In 
the other case, the changes resulted from the implementation of a system for institutional 
planning and improvement that grew out of a 1998 planning retreat, spurred by a president who 
has for nearly ten years been pushing the college to become more student-focused. In making 
these changes, the colleges may also have been responding to the increased attention that higher 
education policy makers in Florida and elsewhere are paying to student outcomes. 

These developments show that colleges can and do change the way they operate, but they also 
suggest that bringing about such changes may require some internal or external catalyst and that 
change generally takes a long time. In every case, the groundwork for the recent changes was 
laid during the study period. In one case, it took nearly a decade for the college’s leadership to 
change the mindset of faculty and staff from a primary focus on access and enrollments to a 
concern for student retention and degree completion as well. 

The fact that the colleges are operating in some substantially different ways three years after our 
study period ended supports our approach of focusing the field research on what the colleges 
were doing during the study period, not what they are doing now. We acknowledge that it is not 
easy to reconstruct, through interviews and document reviews, a college’s policies and practice 
during a period that started eight years prior. Still, to have focused our field research on the 
colleges’ current practices while using data from an earlier period to select the colleges would 
have produced misleading findings. 

This study shows that comparing the performance of different institutions is complicated and 
should be approached with caution. Straightforward comparisons of institutional performance are 
misleading because each college serves a different mix of students and has different 
characteristics, such as size, level of resources, and program mix, that bear on performance. In 
this study, we used a rich set of longitudinal data on cohorts of first-time students to control for 
the effect of individual student characteristics and behaviors on student outcomes. Our purpose 
in examining the relative performance of institutions was to set up fieldwork designed to identify 
the policies and practices that distinguish community colleges that have a higher impact on the 
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success of students (in this case of minority students) from those that have a lower impact. 
Nevertheless, a better benchmark for a college seeking to gauge whether it is doing well or needs 
to improve is probably its own historical performance, rather than the performance of other 
institutions. 



Limitations of the Study 

We believe that the methodology used to select institutions for fieldwork is generally robust, but 
a major limitation is the lack of a direct measure of student socioeconomic status (SES), such as 
family income or parental education, which other studies have shown to be a key determinant in 
college success. SES is correlated with test scores, which we do measure. Also, we selected and 
ranked the colleges using the sub-sample of African American and Hispanic students. We know 
from studies of national survey data on community colleges that students from both populations 
graduate and transfer at lower rates than do Whites, even after controlling for income. Still, 
without data to control for student SES, it could be the case that our methodology ranks some 
colleges higher (or lower) than others because they are serving more (or less) well-off students. 

In addition, while we did find through the field research that the high-impact colleges tended to 
follow more of the policies and practices we hypothesized to be associated with greater 
institutional effectiveness than did the low-impact colleges, these findings should be considered 
suggestive given the small sample size for our qualitative fieldwork. We hope that future studies 
will follow a similar methodology, but with a larger sample of colleges. 
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